rustc-ap-rustc_lexer 654.0.0

Low-level Rust lexer. Tokens produced by this lexer are not yet ready for parsing the Rust syntax, for that see `librustc_parse::lexer`, which converts this basic token stream into wide tokens used by actual parser. The purpose of this crate is to convert raw sources into a labeled sequence of well-known token types, so building an actual Rust token stream will be easier. Main entity of this crate is [`TokenKind`] enum which represents common lexeme types.